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As  a  result  of  characterizing  Joint  scene  content  in 
terms  of  the  joint  probability  density  function,  a  new 
registration  metric  was  defined  as  the  thresholded  difference 
(TD)  method.  It  produced  a  sharper  correlation  peak  than 
other  pixel-by-pixel  methods  investigated.  Analytical 
comparisons  and  simulations  were  done  to  show  the  effects  of 
scene  content  on  registrations  by  direct  cross  correlation, 
mean  absolute  difference  and  thresholded  difference  methods. 

An  adaptive  binary  quantizer  was  implemented  which  sets 
the  quantization  threshold  as  the  mean  or  median  in  a  3  x  3 
moving  window.  It  showed  far  superior  peak  sharpness  when 
compared  with  a  global  thresholding  method;  however,  it 
showed  greater  sensitivity  to  noisy  images. 

Gradient  pre-processing  to  enhance  image  high  frequency 
content  produced  sharper  correlation  peaks  for  all  methods 
except  the  TD  method  which  was  unaffected  but  still  had  the 
sharpest  correlation  peak  of  any  method. 

Simulations  were  done  for  imagery  representing  different 
spectral  bands,  different  time  of  day  and  different  scale. 

For  these  non-compatible  images,  the  TD  method  showed  results 
which  were  generally  as  good  or  better  than  the  other  methods. 
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PREFACE 


This  final  technical  report  describes  the  objectives, 
methods  and  results  of  contract  AFOSR-80-0275  on  "Adaptive 
Correlation  Concepts  for  Non-Compatible  Imagery"  and  covers 
the  period  15  Aug  1980  to  15  Aug  1981.  This  effort  was 
conducted  by  the  Gas  Diagnostics  Research  Division  of  The 
University  of  Tennessee  Space  Institute,  Tullahoma, 

Tennessee  37388.  It  was  sponsored  by  the  U.S,  Air  Force 
Office  of  Scientific  Research,  Bolling  AFB,  DC  20332. 
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INTRODUCTION  AND  SUMMARY 


A  number  of  Air  Force  systems,  whose  operating  regimes 
include  mid-course  and  terminal  phases,  have  need  for  reliable 
stand-alone  guidance  capability.  A  useful  technique  for 
accomplishing  this  guidance  has  been  to  correlate  imagery 
from  an  on-board  sensor  with  reference  imagery  stored  in  a 
computer . 

The  development  of  smaller,  faster  and  less  expensive 
signal  processing  hardware  makes  it  practical  to  consider  in 
more  detail  the  possibility  of  real-time  or  near  real-time 
processing  of  two-dimensional  data  sets  such  as  obtained  from 
sensors  in  multiple  spectral  bands.  Further  it  is  possible  to 
consider  doing  more  of  this  processing  on-board  to  reduce 
dependence  on  data  links.  Applications  for  real-time  two- 
dimensional  processing  of  imagery  include  image  registration, 
automatic  target  handoff,  pattern  recognition  and  feature 
extraction.  The  major  technique  for  accomplishing  these 
applications  is  computation  of  the  cross-correlation  function 
between  a  reference  image  and  the  incoming,  unclassified 
imagery . 

The  remainder  of  this  section  provides  a  summary  of  the 
problem  statement  and  obj ect ives, details  of  the  effort  pro¬ 
posed  to  accomplish  those  objectives,  and  a  summary  of  signifi 
cant  results.  Further  details  of  the  method  and  results  are 
given  in  sections  2.  through  6. 
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1,1  Problem  Statement  and  pv~  j  -  ^  u  ±  ves  .  The  correlation 
technique  works  well  under  ideal  conditions,  i.e.,  compatible 
sensors,  no  geometric  distortion,  high  signal-to-noise  ratio 
and  relaxed  time  constraint  for  computation.  In  fact,  a  numte 
of  variations  on  the  cross-correlation  function  algorithm  have 
been  shown  to  reduce  false  peaks,  provide  high  spatial  regis¬ 
tration  accuracy  or  reduce  required  computation  time.  Unfor¬ 
tunately  the  advantages  mentioned  above  are  usually  competitiv 
and  not  attainable  with  any  one  given  algorithm.  In  many 
cases  the  two  images  are  obtained  from  different  sensors  so 
that  differences  in  f ield-of-view,  spectral  response  and 
geometric  distortion  add  to  the  processing  complexity  and 
introduce  additional  error  sources  in  the  cross-correlation 
technique . 

Variations  on  the  classical  cross-correlation  function 
definition  have  been  examined  which  seek  to  improve  resolution 
reduce  false  peaks  and  increase  processing  speed.  A  technique 
known  as  the  sequential  similarity  detection  algorithm  [i] 
increases  processing  speed  by  computing  a  function  based  on 
the  difference  of  data  point  amplitudes  instead  of  their 
product.  Since  subtraction  is  a  faster  process  than  multipli¬ 
cation,  a  time  savings  is  realized.  A  phase  correlation 
technique  which  improves  resolution  and  false  peak  rejection 
is  described  by  Kuglin  and  Hines  [2]-  Unfortunately  this 
improved  performance  is  achieved  at  the  cost  of  increased 
processing  complexity. 


Algorithms  which  sacrifice  gray-scale  for  increased 
processing  speed  are  reported  by  various  companies  engaged 
in  two-dimensional  correlation  research.  These  algorithms 
offer  the  best  promise  for  real-time  correlation  of  two 
images.  In  fact  if  an  extreme  coarse  quantization  (2 
level  or  1  bit)  is  used  then  multiplication  and  addition 
are  reduced  to  simple  logic  functions.  The  sequential 
similarity  detection  algorithm  and  direct  cross-correla¬ 
tion  algorithms  are  then  computationally  equivalent,  the 
only  difference  being  in  the  specific  logic  operation  re¬ 
quired  for  implementation. 

The  effects  of  coarse  quantization  on  correlation 
processes  have  been  examined  in  detail  elsewhere  [3]  -  [7]> 

A  brief  summary  statement  of  the  results  indicates  that  the 
statistical  disadvantage  incurred  by  coarse  quantization 
of  the  images  is  offset  by  being  able  to  include  more  pixels 
in  the  correlation  computation  for  a  real-time  requirement, 
assuming  that  geometric  distortion  is  not  a  problem.  Thus 
for  an  application  which  requires  real-time  correlation, 
quantization  of  the  data  to  two  levels  is  an  attractive 
technique.  A  more  complete  comparison  of  various  correlation 
techniques  for  the  automatic  handoff  problem  is  given  in  [6] . 

Examination  of  the  probabilities  of  false  registration 
and  detection  for  correlation  of  two  television  images,  as 
a  function  of  the  video  signal-to-noise  ratios  and  a  preset 
correlation  threshold,  has  been  done  for  coarse  quantization 
of  the  video  signals  to  two  levels  [8]. 
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Several  approaches  to  the  false  acquisition  problem 
have  assumed  gaussian  statistics  for  the  correlation  func¬ 
tion  itself.  Roekmore  [9]  extends  this  approach  using  an 
Edgeworth  series  expansion  so  that  the  correlation  function 
statistics  are  no  longer  restricted  to  being  gaussian. 

Limited  parametric  analyses  have  been  done  by  various 
investigators  which  show  the  effects  on  correlator  perfor¬ 
mance  of  quantization  method  [7]  ,  image  rotation,  image  scale 
differences,  synchronization  error  [10]  and  image  sampling 
masks  for  specific  applications.  These  specific  applications 
typically  involve  correlation  of  images  from  like  sensors; 
and,  the  correlator  performance  is  strongly  influenced  by 
individual  scene  statistics.  A  recent  effort  [ll]  has  looked 
at  the  problem  of  correlating  a  day  TV  image  with  imagery 
from  a  co-located  forward-looking  infrared  (PLIR)  system. 

Such  an  approach  to  correlating  imagery  from  different  sensor 
types  is  important  for  development  of  an  all-weather  capabil¬ 
ity  without  requiring  that  a  reference  image  be  stored  for 
each  sensor.  Individual  scene  statistics  can  be  shown  to 
strongly  affect  the  probability  of  correct  registration  for 
any  correlation  algorithm.  Some  method  for  adjusting 
correlator  thresholds  or  preprocessing  steps  adaptively, 
dependent  on  the  scene  statistics,  is  needed. 

It  is  desirable  to  provide  a  sound  theoretical  base 
which  shows  dependence  of  correlator  performance  on  all  the 
above  parameters  in  such  a  way  that  the  results  could  be 
applied  to  new  guidance  concepts,  including  those  using 


non-compatible  sensors  in  an  all-weather  system.  Further, 
it  is  desirable  to  characterize  ability  of  the  correlator  to 
automatically  identify  target  classes  and  to  reject  similar 
false  targets.  Also,  experimental  validation  of  the  theory 
by  correlator  simulation  using  actual  multi-spectral  imagery 
is  desirable. 

Based  on  familiarity  with  guidance  concepts  and  sensors 
used  in  Air  Force  systems  the  following  three  objectives  are 
stated  for  this  effort. 

Objective  # 1:  Develop  specific  characterization  of 
parametric  effects  on  correlator  performance  in  a  way  to 
provide  maximum  flexibility  in  application  to  existing  and 
hew  system  concepts.  Parameters  will  include  but  not  be 
limited  to:  quantization  method,  error  sources,  contrast 
reversals,  threshold  for  quantization,  threshold  for  correla¬ 
tion,  non-compatible  sensors  and  effect  of  selected  pre¬ 
processing  algorithms. 

Objective  #2:  Characterize  dependence  of  correlator 
performance  on  individual  scene  statistics  for  compatible 
and  multi-sensor  imagery.  Develop  theory  for  an  adaptive 
correlator  which  optimizes  performance  for  given  scene 
statistics . 

Objective  #3:  Using  actual  imagery,  validate  parametric 
dependence  through  computer  implementation  of  various  correla¬ 
tion  and  pre-processing  algorithms. 


1.2  Proposed  Effort .  Of  the  related  areas  essential  to' 


the  development  of  an  all-weather  capability  for  guidance 
cased  on  correlation  of  imagery,  several  are  in  need  of 
further  analysis  and  verification  through  simulation  to 
improve  correlator  performance  and  to  predict  critical  para¬ 
metric  dependence. 

Several  key  points  need  to  be  examined  and  understood 
to  provide  a  good  measure  of  expected  correlator  performance 
for  stand-alone,  all-weather  systems.  These  key  points  and 
several  related  problems  will  be  examined  in  detail  in  the 
proposed  program. 

Specifically,  solutions  of  the  following  problems  are 
critical  to  meeting  the  desired  objectives  for  an  image 
correlation  system: 

1.  Scene  Dependence  -  Relative  sharpness  of  the 
correlation  function,  probability  of  correct 
registration  and  the  success  of  pre-processing 
algorithms  have  been  shown  through  simulation 
studies  to  depend  strongly  on  scene  character¬ 
istics.  This  dependence  must  be  characterized 
to  allow  optimization  of  correlator  performance. 

2.  Threshold  Effects  -  Both  in  terms  of  quantizer 
threshold  and  detection  threshold,  there  are 
still  uncertainties  on  how  these  values  should 
be  chosen.  Further,  thresholding  for  edge 
detection  algorithms  has  a  critical  impact  on 
correlator  performance.  A  sound  theoretical 


base  backed  up  by  simulation  results  is  needed 
to  provide  guidelines  on  threshold  selection. 

3-  Pre-processing  and  Distortion  -  Further  work  is 
needed  to  determine  the  effects  of  distortion 
and  what  kinds  of  pre-processing  would  improve 
correlator  performance.  Specifically,  in  terms 
of  images  from  non-compatible  sensors  there  is 
a  need  to  investigate  algorithms  in  addition  to 
the  gradient  method.  A  trade-off  of  correlator 
improvement  versus  computational  burden  must  be 
developed  for  uniform  comparison  of  processing 
methods . 

4.  Adaptive  Correlation  -  Once  parametric  dependence 
for  distortion,  scene  statistics,  threshold 
effects  and  processing  methods  have  been  deter¬ 
mined,  it  is  desirable  to  provide  an  adaptive 
capability  for  the  correlator  to  optimize 
performance  . 

5.  All-weather  Capability  -  All  the  above  analyses 
and  simulations  should  be  based  on  conditions 
which  would  provide  a  stand-alone  system  with 
all-weather  capability. 

The  following  tasks  are  proposed  for  accomnl  1  or 

the  goal  of  developing  a  model  and  methods  for  evaluating  the 
parametric  dependence  of  image  registration  performance. 


Task  1  :  Gain  familiarity  with  specific  details  of 
guidance  concepts  and  sen. -or::  used  in 
existing  and  planned  Air1  Force  .Gy stems . 
This  task  will  re  accomplished  Iv  visits 
to  Wright  Patterson  AFB. 

Task  2:  Example  imagery  for  correlation  analysis 
will  be  selected  from  available  imagery 
from  specific  Aii*  Force  systems,  from 
multi-spectral  photographs  digitised  by 
scanning  microdensitometer,  or  from  com¬ 
puter-generated  special  case  imagery  of 
known  statistics. 

Task  3 :  Distortion  including  rotation  and  scale 
errors  of  selected  magnitudes  will  be 
applied  to  the  imagery.  Effect  or.  ccrreia 
tor  performance  will  be  quantified. 

Task  4:  Quantisation  effects  on  correlator  per¬ 
formance  will  be  evaluated  for  continuous 
gray  scale,  binary  and  3-level  quantisers 
showing  effects  of  quantiser  tr.r«-.u.e  ids  . 

Task  3:  Pre-processing  algorithms',  which  extract 

edge  Information  from  tin-  inn,-.1  ry  will  :  < 
evaluated  in  terms,  of  their  •  f.Y  ■'  -■n 

correlate!-  performance  measures  . 

Task  6:  Scene  characterization  in  terms  of  .’-'in*. 

probability  density  function,  wll]  :> 
eum(  uti-ii  for  tic  samj  Le  ima  f  r  .•  •* 


i  i  s. f. art,  i  ons.  and  (  t"  --j  ret 


Other  descriptive  features  v:ill  be  computed 
for  scene  characterization. 


Task  1:  A  sound  theoretical  bane  will  be  estab¬ 
lished  for  evaluating  correlator  perfor¬ 
mance  in  terms  of  scene  characteristics, 
quantization  method,  selected  distortions 
and  pre-processing  methods. 

Task  8:  Based  on  parametric  analyses  of  correlator 
performance,  adaptive  methods  for  perform¬ 
ance  optimization  will  be  recommended. 

Figure  1.1  shows  the  concepts  included  in  this  proposed 
effort.  The  combined  experimental/theoret ical  approach  should 
provide  definitive  answers  about  the  effects  of  specific 
parameters  on  correlator  performance. 

1,3  Results  Summary.  Early  emphasis  was  placed  on  character¬ 
izing  scene  content  and  on  utilizing  scene  content  to  adap¬ 
tively  modify  the  correlation  approach  (in  choice  of  algorithm, 
quantization  thresholds  or  preprocessing  steps)  .  Specific 
accomplishments  are  summarized  below. 

l.j.l  Thresnolded  difference  algor  it  ijn.  As  a  result  of 
characterizing  Joint  scene  content  in  terms,  of  the  Joint 
probability  density  function  a  new  registration  metric  was 
defined  (referred  to  in  this  report  as  the  threshold- d 
difference  (Tb)  method)  which  p  rov  ides  a  sharper  o-.rr<  Jat  1  en 
peak  than  any  other  j  ixel-by-j  1x^1  method  invest.  !  ga*  •  :  . 

Because  of  this  pr-  mining  performance  th«.  effort  was  .»•••  -  t-  i 

to  provide  a  more  thorough  analysis,  of  the  T!  r. -  t  i .  a. 


r 


1 


EXAMPLE  IMAGERY 

•System  Specific 
•Generated 
•Special  cases 


(QUANTIZATION  &  PRE-PROCESSING 


•  Continuous  gray  scale 


1  Binary 


•  3- level 


r 


•Edge  detection-algorithms 
and  threshold  effects 

•  Pre-distortion 

•System  specific 


SCENE  CHARACTERIZATION 

•  Joint  probability 
density  functions 

•  Feature  extraction 

&  scene  classification 


DISTORTION 

•Rotation  -  selected 
angles  in  azimuth  & 
elevation 

•Scale-5%,  10%,  15%,  20% 
•Pixel  mismatch  probability 


PARAMETER  MODIFICATION 


ADAPTIVE  MODIFICATION 

•  Performance 

•  Scene  depen 

•  Quantizatioi 

•  Distortion 

•  Pre-process: 

•  Recommended 
methods 

bounds 

dence 

i  effects 

dependence 

Lng  effects 
adaptive 

_ _ 

REGISTRATION  PERFORMANCE 

•Probability  of  false  fix 

•  Probability  of  registratior 

•  Statistical  reliability 

•  Registration  accuracy 


PARAMETRIC  DEPLNDENC 


Figure  1.1  Concepts  for  correlator  evaluation 


lU 


1.3.2  Gradient  preprocessing.  Since  images  with  higher 
spatial  frequency  content  produce  sharper  correlation  peaks, 
it  follows  that  a  high  frequency  emphasis  processor  such  as 
the  gradient  should  produce  a  sharper  correlation  peak.  A 
simple  3x3  gradient  processor  was  applied  to  typical  TV 
and  FLIR  imagery.  The  resultant  correlations  were  sharper 
for  all  algorithms  except  the  TD  method  which  was  still  the 
sharpest  peak  of  all.  Since  the  TD  method  is  inherently  a 
high  resolution  method  '-niywa'%  then  gradient  processing  had 
little  or  no  effect  on  :  *■  , 

1.3.3  Adaptive  quantization  threshold.  For  binary  and 
tri-level  correlators  the  quantization  thresholds  are  typically 
chosen  to  be  dependent  on  global  scene  statistics  (mean  and 
standard  deviation) .  This  global  thresholding  tends  to 
reduce  high  spatial  frequencies  in  the  image,  even  for  images 
which  have  been  gradient  processed.  A  locally  adaptive  quan¬ 
tization  method  was  implemented  whereby  thresholds  are  based 

on  statistics  (mean  and  standard  deviation)  within  a  small 
window  about  the  pixel  being  quantized.  Simulation  results 
for  a  binary  correlator  using  an  adaptive  threshold  in  a 
3x3  window  gave  a  high  resolution  correlation  function. 

2.  SCENE  CHARACTERIZATION 

One  objective  for  this  effort  was  to  develop  methods 
for  characterizing  scene  content  and  to  make  use  of  knowledge 
of  scene  content  to  optimize  registration  algorithms,  para¬ 
meters,  etc  . 
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2.1  Scene  Content  and  Correlation  Accuracy.  One  measure 
of  scene  content  is  its  two-dimensional  spatial  transform 
which  gives  a  detailed  measure  of  spatial  frequency  content 
in  the  image.  Prom  two  images  r  and  s,  whose  transforms  are 

R  and  S,  we  form  the  two-dimensional  cross  spectrum  of  spatial 
frequency . 

Grs(6,3)  =  R(6,3)  S*(B,3)  (2-1) 

The  inverse  transform  of  G  is  the  cross  correlation 
function,  C,  of  r  and  s. 

Crs(a,6)<=|=^>GRS(B,3)  (2-2) 

Further,  because  of  the  inverse  relationship  of  size 
between  two  Fourier  domains,  it  follows  that  high  spatial 
frequencies  in  G  transform  to  sharp  correlation  peaks  in  C. 
Therefore,  a  sharper,  more  accurate  correlation  peak  is 
expected  from  imagery  with  high  spatial  frequency  content; 
and,  preprocessing  algorithms  which  enhance  high  frequencies 
have  the  potential  for  improved  correlator  performance. 

2.2  Statistical  Characterization.  One  method  for  concise 
characterization  of  scene  content  is  in  terms  of  statistical 
descriptions.  Univariate  statistics  give  a  description  of 
individual  images  and  bivariate  statistics  provide  joint 
descriptions  between  images.  In  either  case,  the  most  conplet 
statistical  description  is  provided  by  estimates  for  the  n-th 
order  probability  density  functions  (pdf).  Other  statistical 
descriptors  such  as  moments  or  statistical  functions  can  be 
defined  in  terms  of  the  appropriate  pdf. 


2.2.1  Simple  moments  and  univariate  statistics.  Moan  and 
standard  deviation  are  often  used  as  parameters  in  setting 
quantization  thresholds  for  binary  and  tri-level  registration 
methods.  These  parameters  are  estimated  as  array  averages 
for  an  image,  f,  with  K  x  L  pixels  as 

Mean  estimate 

£  -  re  E  Ef(1'J)  (2'3) 

i=l  j=l 

Standard  deviation  estimate 

8  ■  [imbrE  E  (f<1>3)-0)2]"  <.2-k) 

L  i=l  j=l  x  J 

Additional  statistical  descriptions  for  the  image  are 
given  by  the  following  functions. 

1.  Histogram  -  an  estimate  for  the  univariate 
amplitude  probability  density  function 
(for  8  bit  quantization) 

h(a)  =  jjg  a  =  0,255  (2-5) 

n&  =  number  of  pixels  at  amplitude  a 

2.  Fourier  spectrum 

Gff(e,a)  =  |fFT  |f  (x,y )  jj  (2-6) 

3.  Autocorrelation  function 

Cff(a,6)  =  IFFT^Gff(6,‘3)J  (2-7) 
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2.2.2  Joint  statistical  description.  Registration  proper¬ 
ties  of  two  images  are  completely  characterized  by  the  joint 
probability  density  function  between  the  two  images  with 
shift  in  two  dimensions  (a,6)  as  parameters.  Parametric  ana¬ 
lysis  of  this  joint  pdf  requires  knowledge  of  the  cross 
correlation  function  between  the  two  images.  Since  the  objec¬ 
tive  is  to  use  the  joint  pdf  to  predict  cross  correlation 
properties,  the  analytical  approach  appears  to  go  in  circles. 

An  estimate  for  the  joint  pdf  between  two  images  is 
given  by  their  joint  histogram.  For  eight-bit  images  the 
joint  histogram  has  a  domain  of  256  x  256  values  and  presents 
a  relatively  complicated  statistical  description  of  the 
joint  properties  of  the  two  images.  Additionally  a  complete 
histogram  surface  is  generated  for  each  relative  spatial 
shift  between  the  two  images.  Thus  it  appears  that  charac¬ 
terizing  two  images  by  their  joint  histograms  is  a  formida¬ 
ble  task  which  produces  too  much  information  to  be  useable. 
Strictly  speaking  this  is  true;  however,  there  is  useful 
information  and  enhanced  understanding  to  be  achieved  from 
examining  the  joint  histogram. 

For  images  r  and  s  with  8-bit  pixel  values  and  size 
K  x  L,  the  joint  histogram  for  shift  (a, 6)  is  given  by 

n(r  ,s  ) 

h(a,b,a,S)  =  — ^ — 2-  a,b  =  0,255  (2-8) 

where  n(raJsb)  =  number  of  pixel  pairs 
wherein  the  amplitude  in  r  was  a 
while  the  amplitude  in  s  was  b. 


ir  achieved 


home  insight  into  the  regi  strat  i on  pfocesi' 
by  examining;  the  properties  of  the  joint  histogram  for  some 
special  cases.  At  registration,  ideally,  all  pixels  match 
exactly  and  all  points  in  the  joint  histogram  lie  along  the 
r=s  line  as  shown  in  figure  2.1a.)  Realistically,  at  registra¬ 
tion  some  pixels  will  not  match  due  to  uncorrelated  noise  in 
the  images  or  due  to  geometric  distortions.  These  mismatched 
pixels  are  represented  as  points  scattered  off  the  r=s  line 
in  figures  2.1b.)  and  c.).  The  mis  scattering  distance  is 
relatable  to  signal- to-noise  ratio  or  to  percent  distortion. 

The  effects  of  gain  errors  and  mean  value  shifts  between 
the  two  images  are  illustrated  in  figures  2.1  d . ) ,  e.)  and  f.). 
It  will  be  shown  later  that  these  two  errors  may  be  corrected 
by  a  simple  amplitude  range  scaling  prior  to  registration. 

Further  insight  into  the  behavior  of  the  joint  pdf  for 
two  images  with  varying  degrees  of  correlation  is  gainc-d  by 
examining  tne  special  case  ’where  r  and  s  are  jointly  gauss  ian. 
hqui-amplitude  contours  in  the  joint  pdf  domain  are  shown  in 
Figure  2.2  for  a)  uncorrelated-equal  variance;  1.) 
unequal  variance;  c)  partially  correlated;  and  d) 

(perfectly  registered). 


uncorrelated- 


a.)  Ideal  registration 


b.)  Uncorrelated  noise  at  registration 


c.)  Geometric  distortion  at  registration 
figure  2.1  Joint  Histogram  Properties 


ct 


3.  Riiu  I  STRATI  O’J  METRICS 


There  are  several  registration  metrics  which  may  be 
defined  for  comparing  two  images  [a]  and  [12]  .  Thru-  metric.: 
were  investigated  and  compared  as  part  of  this  effort.  The 
"direct  cross  correlation"  and  "mean  absolute  difference" 
metrics  were  included  because  of  their  widespread  use  in 
real-time  or  near  real-time  applications.  The  "thresholded 
difference"  metric  was  developed  as  part  of  this  effort. 
Definitions  and  properties  for  these  three  metrics  are  given 
in  this  section. 

Definitions  are  given  for  correlating  two  images  r  and  s 
of  size  K  x  L  and  i'i  x  N  pixels  respectively  as  shown  in 
Figure  3.1.  Shift  values  (p,q)  are  restricted  so  that  the 
images  always  have  KL  overlapping  pixels. 

3.1  Direct  Cross  Correlation.  The  definition  for  the  discrete 
cross  correlation  between  two  spatial  functions  f^(n,m)  is  given 
by  (3-D 

C(p,q)  =  E  |f1(n,m)  f2(n+p,  m+q)J  (3-1) 

where : 

E  £  •  J  is  expected  value 

Based  on  an  image  model  which  is  characterized  by  a  shift- 
variant  mean  and  shift-invariant  second-order  statistics  [l Z]  , 
(3-1)  may  be  estimated  from  a  spatially  averaged  function. 
Equation  (3-2)  defines  a  nomalized  direct  cross  correlation 
( DCC )  algorithm  for  estimating  C(p,q)  for  the  image  models 
shown  in  Figure  3 • 1 • 


iy 
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Crs{P,q) 


K  L 
^  ^  ]  r(m,n)s(m+p,n+q) 

m=l  n=l 


K  L 


ZZ  r,i!<"‘,n’ 


m=l  n=l 


r  K 


ZZ' 

m=l  n=l 


where:  p  =  0,  M-K 

q  =  0,  M-L 

For  quantization  of  r  and  s  to  1  bit  (2  levels)  or  ^1 , 
the  denominator  term  in  (3-2)  reduces  to  KL  to  give 


Crs(p,q)  =  prj-  r (m,n) s (m+p ,  n+q) 

m=l  n=l 

Both  (3-2)  and  (3-3)  have  the  property  that  $  =  1  at 
registrat ion . 

3.2  Mean  Absolute  Difference.  The  definition  of  the  mean 
absolute  difference  algorithm  (MAD)  for  the  images  in  Figure 
3.1  is  given  by  equation  (3-*0. 


A 

E 


rs 


( p j q) =I  Z 


m=l  n=l 


r(m,n)  -  r  -  s( m+p, n+q)  +  ? 


cc 


where  r  is  the  mean  value  of  r 

K  L 


r  = 


KL 


z  z  r (m,n) 


m=l  n=l 


and  Spq  is  the  mean  value  of  KL  pixels  in  s  at  shift  p,q 


s  = 


=  ^z  z 

m=l  n  =  l 


s (m+p, n+q ) 
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A  more  logical  approach  Is  to  consider  two  pixels  as 
matched  if  their  values  agree  within  some  specified  threshold 
centered  about  the  r  =  s  line.  The  result  is.  shown  in  Figure 
3.3a)  in  terms  of  the  Joint  pdf  and  a  simple  implementation 
is  shown  in  Figure  3-3  t). 

The  thresholded  difference  algorithm  is  computationally 
equivalent  to  the  S-bit  MAD  algorithm  except  that  the  magni¬ 
tude  eri'or  is  thresholded.  Tills  allows  for  small  (specifiable) 
errors  between  amplitudes  of  r  and  s  with  no  penalty.  The 
resultant  metric  should  exhibit  a  sharper  registration  peak. 
Simulation  results  presented  in  section  5-  verify  tills  hypcthcv. 

=  s 


a)  Representation  in  joint  pdf  domain 

r  ( x , 
s(x. 


Figure  3 • 3  Thresholded  Difference  Method 
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4.  prfprocf.;::ikg  and  quantization 

There  are  numerous,  proprocer-s  inn  net  hods  with  f  otei.tial 
for  application  to  the  registration  problem.  For  this  effort, 
methods  used  in  simulations  were  kept  simple  and  selected 
with  the  goal  of  improving  correlation  peak  sharpness.  Rela¬ 
tively  simple  statistically  based  thresholds  were  defined  for 
the  binary  and  tri-level  quantizers. 

Gradient  preprocessing  was  implemented  to  enhance  high 
frequency  content  in  the  images  and  produce  a  sharper  correla¬ 
tion  peak.  A  locally  adaptive  method  was  implemented  for  the 
binary  quantiser. 

Preprocessing  to  correct  for  mean  value  shift  and  gain 
differences  for  pixel  amplitudes  for  the  two  images  is  impor¬ 
tant  if  good  results  are  to  be  obtained  with  the  throsholded 
difference  method. 

These  preprocessing  methods  are  discussed  in  this  section 

4.1  Binary  and  Tri-Level  Quantization.  The  images  were 
quantized  to  two  levels  or  three  levels  as  shown  in  Figure  4.1 


BINARY 


Binary  and  Tri -lowl  Quantizers 


Figure  4.1 


For  the  images 


thresholds  were  defined  in  terms  of 


statistics  for  the  total  (global)  image.  For  binary  quanti¬ 
zation,  T  was  chosen  as  the  median  value  in  the  larger  image. 
This  choice  for  T  gives  an  equal  number  of  plus  and  minus 
ones  in  the  larger  image.  For  ehe  tri-level  quantization, 
choosing  as  the  lower  1/3  and  as  the  2/3 
level  in  terms  of  total  pixels  in  the  amplitude  histogram 
(ie  33  and  67  percentiles)  gives  an  equal  number  of  pixels 
at  -1,  0  and  +1  for  the  output  quantized  image.  In  summary 
the  following  thresholds  were  used. 

T  =  median  (binary) 

T1  '  33  !'ei,centlle((tri-level) 

=  67  percentile) 

Alternate  choices  were  available  and  are  Included  in 
the  program  software.  These  include 

T  =  y  ±  ko  I 

T!  2  =  M  ±  ko  (  (4-1 

T^  2  =  l°w  % >  high  %  I 

where  u  is  the  mean,  o  is  standard  deviation  and  k  is  a 
specifiable  constant. 

4.2  Adaptive  Binary  Quantizer.  A  locally  adaptive  binary 
quantizer  was  implemented  which  results  in  a  quantized  image 
that  retains  the  image  high  frequency  content.  The  r 

pixel  in  a  3  x  3  array  is,  quantized  dependent  on  a  s.tv.  sM  cn 
threshold  calculated  within  that  window.  As  t.h>-  win  i  w 
moves  across  the  image-  the  thro  -.he. Id  Is  rood  1  f  I •  i  ami  d  no 
quantize  the  roidule  pixel  in  th<  window. 


.  t. 


L’pec  if  ical  ly 


T  =  median  in  a  j  x  5  window. 

Thin  adaptive  method  retains  the  image  high  frequency 
content  and  should  result  in  a  sharper  correlation  peak  for 
the  DCC  or  MAD  algorithms.  Further  it  has  the  advantage  of 
relatively  simple  implementation  in  real  time,  especially  if 
the  mean  instead  of  the  median  is  used  as  the  threshold . 
Simulation  results  in  section  5  will  compare  correlation 
sharpness  for  the  global  and  locally  adaptive  quantizers. 

14 . 3  Gradient  ['recessing.  Gradient  preprocessing  of  the  imai 
prior  to  correlation  offers  two  potential  advantages .  First 
the  gradient  operation  enhances  high  fr-.  quency  content  and 
should  result  in  a  sharper  correlation  peak.  Secondly , 
gradient  processing  emphasises  edge  features  in  an  imaee, 
which  may  be  the  only  features  of  similarity  among  different 
spectral  band  images  of  a  given  scene. 

There  are  numerous  gradient  processors  from  which  to 
choose  including  the  Roberts  cross  operator,  I /at  in clan  ana 
convolutional  windows  of  varying  geometries  and  scoff  I  ?  i*  :.t 
values.  Because  of  its  simplicity  a  frewitt  r.arr.3  ru-.u  gvadi  • 
operation  was  applied  to  the  images,  prior  to  -mart  1  sat  i on 
and/or  correlation.  The  gradient  is  calculated  fr  r 


= 

I  g  ( m  ,  n ) 

+ 

|g  ,  ( m , n  ) 

l  x 

» 

f 


where  g^(m  ,n)  results  from  convolving  images  r  and  .•  wi-.h 
window  Wj  and  g^(m,n)  results  from  convolution  with  window  v.v . 
1' h e  s e  w  i nd o w s  were 
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-1 
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1 
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4.4  Gain  and  Fean  Correction.  The  threchclded  difference 

method  is  especially  sensitive  to  errors  in  mean  value  or 

gain  between  two  images  to  be  correlated.  This  was  illustrated 

in  Figure  2.1  d)  e)  and  f)  where  it  is  clear  that  these  errors 

(even  at  registration)  shift  values  off  the  r=s  line  in  the 

joint  pdf  domain.  Both  these  problems  are  eliminated  if  r  and 

s  images  are  first  scaled  to  the  same  amplitude  range,  provided 

that  neither  image  function  is  clipped.  Consider  a  function, 

r,  whicn  is  linearly  represented  over  the  range  r_  ,  to 

r  and  a  function,  s,  which  is  linearly  represented  over 

the  range  s  .  to  s  as  shewn  in  Figure  4.2. 
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This  rescaling,  wan  up,  lied  as  a  necessary  pr*--:  re Ceasing 
step  prior  to  the  tnrecholdvd  difference  con-elat  icn  algorithm. 
The  rescaling  is  accomplished  using  equations  (4-5)  and  (4-0)  . 


r-r  ^ . 

r(re-scaled)  =  - ^2 —  *155  (4-5) 

max  min 

s  —  s 

s(re-scaled)  =  — — —  *  255  (4-6) 

max  °min 

For  the  case  of  uncorrelated  noise  in  r  and  s,  the  linear 
re-scaling  applied  to  signal  plus  noise  will  force  pixels  that 
are  matched  to  lie  off  the  r=s  line.  For  the  thresholded 
difference  method  to  work  in  this  case  requires  that  the 
threshold  be  increased  to  allow  larger  differences  between  r 
and  s.  Further  the  threshold  should  be  related  to  the  ms 
noise  levels  in  the  two  images. 

5.  SIMULATION  RESULTS 

All  algorithms  were  first  tested  using  simple  input 
arrays  of  data  for  which  the  processing  result r  wore  predic¬ 
table.  Next  the  various  registration  metrics  were  up:  2  led  to 
actual  electro-optical  imagery.  The  results  of  those  sir:. dilu¬ 
tions  are  described  in  this  section.  A  summary  of  t.hc  simula¬ 
tions  dune  is.  given  in  Figure  5.1.  Over  120  sin  diet  he.  runs 
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Figure  5.1  Simulation  Options  Summary 
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metrics.  The  effects  of  noise,  scene  content,  time  of  day 
variations,  different  spectral  bands  and  different  scale 
are  illustrated  by  the  simulations.  Further,  the  effects  of 
binary,  tri-level  and  8-bit  quantization  as  well  as  gradient 
pre-processing  and  an  adaptive  binary  quantizer  are  shewn  by 


selected  simulations. 

5-1  Scene  Descriptions.  Imagery  for  use  in  the  correlator 
simulations  was  obtained  from  two  sources: 

1.  the  U.  S.  Army  Missile  Command,  Redstone  Arsenal, 
Alabama;  and 

2.  the  U.S.  Air  Force  through  The  Analytic  Sciences 
Corporation  (TASC). 

Both  data  sets  contain  multi-spectral,  low  altitude,  short 
range  imagery  from  co-located  sensors. 

Ten  image  files  were  established  from  the  data  tapes  for 
input  to  the  simulation  program.  A  brief  description  of 
these  images  is  given  in  Table  5.1. 

The  first  four  images  were  used  in  auto-correlation 
calculations  to  provide  a  known  registration  point  for  com¬ 
paring  algorithms  and  quantization  methods.  To  accomplish 
this,  reference  and  sensed  images  were  established  as  shown 
in  Figure  5.2.  The  reference  image  is  extracted  as  the 
64  x  64  center  portion  of  the  larger  128  x  128  sensed  image. 
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p ,  q  =  0,64 
( p  * , q  * )  =  (32,32) 

r  =  reference  image 
s  =  sensed  image 


Figure  5.2  Image  Segments  from  Silicon  TV 

Registration  then  occurs  at  a  shift  position  (p*,q*)  =  (32,32). 

For  Ohio  Files  2  and  10  the  geometry  shown  in  Figure 
5.3  was  used.  The  reference  image  was  selected  as  the  central 
50  by  50  pixels  from  File  2.  This  image  was  then  correlated 
with  the  larger  150  by  150  sensed  image  (File  10) . 
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Figure  5.3  Ohio  Files  2  and  10  Image  Segments 


For  Ohio  Files; 


26  and  ?7  the  image  geometries  shown  in 
Figure  5.4  was  used.  The  reference  image  was  taken  as;  the 
lower  right  quadrant  of  file  26  (^0  x  64  pixels).  This 
segment  was  correlated  with  the  entire  (80  x  128  pixels) 
File  27  image. 
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p  =  0,41 
q  =  0,65 

(p*,  a*)  =  (41,65) 


Figure  5.4  Ohio  Files  26  and  27  Image  Segments 

For  the  HSV  Files  9  and  10,  there  was  a  difference  in 
scale  as  well  as  spectral  band.  An  area  of  commonality  was 
selected  visually  from  photographs.  Scale  difference  was 
computed  from  dimensions  of  a  large  rectangular  target  board 
in  the  two  images.  Since  File  10  was  a  narrow  f ield-of-view 
(NFOV)  image,  objects  in  File  10  appeared  larger  than  in  File 
9  by  a  ratio  determined  to  be  1.0/0.55.  Scaling  was  applied 
to  reduce  the  size  of  objects  in  File  10  to  match  the  smaller 
size  in  File  9-  After  scaling.  File  10  then  had  only  233  by 
215  pixels.  Figure  5.5  shows  the  relative  geometries  for 
image  segments  from  HSV  Files  9  and  10.  A  56  x  75  1  °orenco 
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Figure  5-5  HSV  Files  9  &  10  Image  Segments 

from  File  10  and  a  100  x  120  sensed  image  is  obtained  from 
File  9  with  a  match  at  shift  (p  ,  q  )  =  (11,6). 

5.2  Metric  Comparison.  Comparison  of  the  direct  cross 
correlation  (DCC),  mean  absolute  difference  (MAD)  and  thres'nolded 
difference  (TD)  metrics  is  given  in  this  section  for  the  noise- 
free  correlation  of  imagery  represented  by  Scene  1  and  Scene 
2  (ie  autocorrelation  of  Silicon  TV  imagery)  as  shown  in 
Figure  5.2.  Results  are  shown  for  8-bit,  binary  and  tri-1  ”e] 


quant izat ions . 


Results  are  shown  in  Figures  p .  6  thru  p .  for  Zc  •  n-.  1 
and  in  Figures  5.10  thru  p.13  lor  Scene  k.  Tne  figur-  s  show 
cross-sectional  plots  through  the  correlation  peak  in  vertical 
and  horizontal  directions  respectively.  Peak  sharpness  is 
greater  in  the  vertical  direction  for  both  images.  This  was 
felt  to  be  a  result  of  the  limited  sampling  rate  applied  to 
the  875  line  video  from  which  the  images  were  obtained.  The 
expected  result  would  be  reduced  frequency  content  (and  a 
broader  correlation  peak)  for  the  horizontal  direction. 

The  comparisons  shown  in  Figure  5.6  are  a)  8-bit  DCC 
(either  direct  or  FFT  implementation),  b)  8-bit  MAD,  c)  binary 
DCC  or  MAD,  d)  tri-level  MAD  and  e)  tri-level  DCC.  All  method 
gave  the  correct  peak  location  of  32  and  peak  amplitudes 
as  expected  from  algorithm  definitions  for  ideal  registration . 
Thus  the  only  feature  for  comparison  is  relative  peak  sharp¬ 
ness.  As  can  be  seen,  there  are  only  slight  variations  in 
peak  sharpness  among  the  five  methods  in  Figure  5.6. 

In  Figure  5-7,  a)  8-bit  DCC  repeated  for  comparison, 
b)  TD  (T=0)  ,  c)  TD  (T-2) ,  d)  TD  (T=5)  and  e)  TD  (T=10) 
simulations  are  shown  for  the  same  conditions  as  Figure  5.6. 
The  thresholded  difference  method  is  clearly  superior  in  terms 
of  correlation  peak  sharpness  if  the  threshold,  T,  can  be 
kept  small.  It  should  be  emphasized  here  that  a  threshold  of 
T=0  gives  almost  perfect  results  for  the  ideal  conditions 
under  which  the  correlation  was  done  ( ie  at  registration  the 
images  were  known  to  be  a  perfect  match).  For  cross-come]  ni.  i 


b)  MAD  8-bit 


c)  BINARY  DCC  or  MAD 
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Figure  5.6  Scene  1, Vertical  Profiler,  Through  I'e.'tk  v  N 
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••.■all  otic  images  it  to  likely  that  a  at::. ft.:*  of  pixels  v;  ill 
vary  la  amplitude  between  tin.*  two  Images  at  reg .  ot  x-at  lor  . 
Exactly  how  much  they  vary  would  depend  on  signal- to-noise 
rat  loo ,  geometric  distortion  and  sensor  anomalies. 

in  comparing  the  threshold  values  shown  in  the  figure, 
cue  should  evaluate  T  as  the  allowable  error  between  two 
gray  levels  which  each  have  a  possible  range  of  £  bits  (0-255) . 

A  comparison  which  .illustrates  that  the  7D  method  is  a 
high  frequency  emphasis  method  is  shewn  in  Figure  5.6  and 
5.9  for  the  peak  profile  in  the  horizontal  direction  for 
Scene  1.  A  very  flat  correlation  peak  (easily  obscured  by 
noise)  is  produced  by  all  methods  except  the  7?  method  which 
produces  a  peak  whose  sharpness  again  depends  on  the  value  of 
T.  If  the  images  are  known  tc  be  of  high  quality  (low  noise, 
low  distortion)  then  it  is  possible  to  achieve  high  correla¬ 
tion  accuracy  with  the  ID  method  (using  a  lew  value  of  7), 
even  if  the  scene  has  low  frequencies  only. 

Additional  comparisons  are  provided  by  Figures  5-10 
through  5-13  which  give  peak  profiles  in  vertical  and  horizon¬ 
tal  directions  for  Scene  2. 

5.3  Noise  Effects  on  Registration  Peak.  To  evaluate  the 
effects  of  uncorrelated  noise  on  the  different  registration 
metrics,  random  noise  was  added  to  r  and  s  on  a  pixel  by  pixel 
basis  and  independently  for  each  image.  Noise-free  r  and  0 
images  were  taken  from  Scene  1  as  indicated  in  Figure  5.2. 
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Zero-mean  random  noise  with  specifiable  standard  deviation 
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added  to  each  image  prior  to  quantization  and  correlation. 

Two  sets  of  simulations  were  done  representing  fairly 
high  noise  levels.  Signal-to-noise  ratios  were  defined  as 

SNR  =  g-  (5-1) 

n 

where  a  is  the  image  standard  deviation  (image  r  or  s)  and 
on  is  the  specifiable  noise  standard  deviation.  For  one  set 
of  simulations  a  SNR  of  0.43  was  used,  and  for  the  other  set, 
a  SNR  of  1.42  was  used.  Example  results  are  given  as  vertical 
profiles  through  the  peak  in  Figure  5.14  for  SNR  =  0.43  and 
Figure  5-15  for  SNR  =  1.42.  Each  figure  compares  a)  8-bit  DCC, 
b)  binary  DCC  or  MAD  and  c),  d) ,  e) ,  f)  TD  for  four  different 
thresholds . 

It  is  clear  that  the  TD  method  (which  is  a  high  frequency 
emphasis  method)  is  more  sensitive  to  the  high  frequency 
(pixel  by  pixel)  noise  added  to  the  images.  It  even  produces 
false  peaks  for  very  low  values  of  T.  Performance  of  the  TD 
method  is  similar  to  the  DCC  method  if  T  is  "large  enough"  . 

The  value  of  T  which  is  "large  enough"  depends  on  noise  level. 
For  the  two  simulations,  the  TD  method  began  to  behave  reason¬ 
ably  well  at  a  threshold  of  T  =  0.5  0  .  This  is  illustrated 

n 

in  Figure  5-14  e)  where  T  =  40  =  .6  (an=65)  and  in  Figure 

5.15  e)  where  T  =  10  =  0.51  a  (o  =  19-5). 

n  n 

Additional  noise  dependence  is  Indicated  in  Table  5  - 2 
which  gives  calculated  peak  locations  for  all  the  algoriliuns 


4o 


tested  for  .Scene  1  and  SNR 


=  0.43  and  1.42.  Note-  that  error 
are  larger  for  the  horizontal  (second)  peak  location.  This 
is  expected  since  high  frequency  noise  was  added  to  an  image 
with  inherent  low  frequency  in  the  horizontal  direction. 


TABLE  5-2  NOISE  EFFECTS 


Correct  Peak  Location  (32,32) 

METHOD 

COMPUTED  PEAK  LOCATION 

SNR=0 . 43 

SNR=1 .42 

DCC ( 8-Bit ) 

(33,32) 

(32.32) 

DCC(Tri-Level 

(31,59) 

(32.37) 

DCC( Binary ) 

(31,59) 

(32,32) 

MAD(8-Bit) 

(33,32) 

(32.32) 

MAD(Tri-Level 

(31,59) 

(33,32) 

MAD (Binary) 

(31,59) 

(32,3 2) 

TD 

(T=0 ) 

(25,51) 

(31,26) 

TD 

(T=2 ) 

X 

(33,38) 

TD 

(T=5) 

X 

(32,35) 

TD 

H3 
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(32,32) 

TD 
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(1 

Eh 

(30,35) 

(32,32) 

TD 

(T=30) 
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(32,32) 
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(33,32) 
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TD 
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(31,40) 
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TD 

(T=80) 

(33,41) 
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(T-100) 

(33,32) 

X 

5.4  Gradient  Preprocessed  Images.  Gradient  preprocessing 
by  application  of  a  3  x  3  Prewitt  operator  to  Cc-.nes  1  and  1 
was  done  prior  to  quantization  and  correlation.  The  resultant 
high-f requency-emphas ized  images  should  exhibit  sharper 
correlation  peaks. 

The  results  are  shown  as  peak  profiles  in  Figure  5.16 
Scene  1,  Vertical;  Figure  5.17  Scene  1,  Horizontal;  Figure 
5.18  Scene  2,  Vertical  and  Figure  5.19  Scene  2,  Horizontal 
for  various  registration  metrics.  Results  without  and  with 
gradient  preprocessing  are  shown  side-by-side  for  easy  com¬ 
parison.  In  every  case  except  for  the  TD  method,  gradient 
preprocessing  did  produce  a  sharper  correlation  peak.  It 
is  postulated  that  sinje  t  .,o  TD  method  exhibits  high  fre¬ 
quency  emphasis  characteristics  itself,  then  high-frequency- 
emphasis  filtering  provided  by  the  gradient  processor  offers 
no  additional  gain.  This  "independence"  of  the  TD  method  to 
gradient  processing  was  observed  for  all  values  of  T  used  in 
the  simulations. 

5-5  Adaptive  Quantizer.  A  locally  adaptive  threshold  was 
used  to  quantize  images  to  1  bit  (binary  quantization)  before 
correlation.  The  threshold  was  computed  as  either  the  mean 
or  median  in  a  3  x  3  moving  window.  At  each  1 scat i on  the 
center  pixel  is  quantized  about  the  threshold  computed  for 
that  location. 

This  adaptive  threshold  iri  effect  emphasis'  s  the  i as" 
high  frequency  content  and  should  result  in  a  much  shar:  «.-r 
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correlation  function .  For  binary  quantization  the  FA D  and 
DCC  correlation  metrics  give  equivalent  results.  Figure  5 .2d 
shows  the  vertical  cross-section  through  the  peak  for1  "cone  1. 
The  a)  part  of  Figure  5.20  gives  the  result  without  adaptive 
thresholding  for  comparison.  The  b)  part  shows  the  result 
for  a  3  x  3  adaptive  mean  threshold  with  no  noise.  Parts 
c)  and  d)  show  the  DCC  and  MAD  adaptive  threshold  methods 
(mean  threshold)  for  additive  noise  (SNR  =  1.42);  and  parts 
e)  and  f)  show  similar  results  for  a  SNR  =  0.43.  Different 
profiles  were  obtained  for  DCC  and  MAD  with  additive  noise 
because  the  noise  was  different  for  separate  runs  (although 
SNR  was  fixed). 

Figure  5-21  shows  peak  profiles  in  a  horizontal  direction 
for  identical  conditions  as  in  Figure  5.20.  The  high  fre¬ 
quency  characteristic  of  the  adaptive  threshold  method  is 
particularly  well  demonstrated  by  a  comparison  of  the  a)  and 
b)  parts  of  Figure  5-21.  Since  noise  added  on  a  pixel  by 
pixel  basis  is  inherently  high  frequency,  the  adaptive 
threshold  method  should  exhibit  increased  noise  sensitivity 
for  the  simulations  with  additive  noise.  This  'was  verified 
by  all  simulations  of  the  adaptive  method  with  additive  noise. 
A  comparison  of  parts  c)  d)  e)  and  f)  of  Figure  5-20  with 
Figures  5-14  and  5-15  b)  illustrates  the  relative  effects  ef 
noise  on  the  adaptive  threshold  method. 

Simulation  results  for  a  3  x  3  adaptive  m-  dian  tine  eh  id 

for  the  vertical  peak  prof!].-  and 
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are  given  in  Figures  5.22 
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Figure  5-20  Scene  1  Vertical,  3x3  Mean  Adaptive 
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Figure  5.23  Scene  1,  Horizontal,  3x3  Median  Adaptive 


^.6.1  Different  Spectral  Bands.  Images  listed  as  Ohio  ril 
2b  ana  27  in  Table  5.1  were  images  of  the  same  scene  but  in 
different  spectral  bands.  Simulations  were-  done  to  compare 
the  DCC,  MAD  and  TD  registration  methods  with  and  without 
gradient  preprocessing.  A  summary  of  the  results  is  given 
in  Table  5-4  in  terms  of  computed  registration  coordinates. 


TABLE  5-4 


Results  for  Different  Spectral  Bands 


Images  7  &  8.  Ohio  File  26  (8-12y) 

Ohio  File  27  (Visible) 


Method 

Registration 

True  Registration  Point 

Point 
=  (41,65) 

DCC  8  bit 

(37,2) 

MAD  8  bit 

(40,29) 

TD  (T=10) 

(41,29) 

TD  (T=50) 

(41,29) 

DCC  8  bit  (Gradient) 

(  2,25) 

MAD  8  bit  (Gradient) 

(  2,51) 

TD  (T=10)  (Gradient) 

(41,50) 

TD  (T=50)  (Gradient) 

(  4,32) 

Clearly,  the  closest  match  is  (41,50)  for  the  TD  (T=10) 
(Gradient)  method.  Yet  the  match  is  not  accurate  in  the 
horizontal  direction.  This  result  indicates  that  further 
preprocessing  to  extract  similarities  between  the  two  image 
is  required. 


The  results  are  shown  as  profiles  through  the  peak  in 
Figures  5.24  and  5.25  for  the  unprocessed  images  and  Figure 
5.26  and  5.27  for  the  gradient  images. 

5.6.2  Different  Time  of  Day.  Images  listed  as  Ohio  Files 
2  and  10  were  infrared  images  taken  at  different  times  of 
day.  These  were  used  in  cross-correlation  simulations. 
Results  are  summarized  in  Table  5-5- 


TABLE  5-5 


Results  for  Different  Time  of  Day 

I  Images  5  &  6.  Ohio  File  2 

(Noon) 

Ohio  File  10  (2:00  PM) 

Method 

Registration 

Point 

True  Match  Point  (51, 

51) 

DCC  3  bit 

(51,51) 

MAD  8  bit 

(51,51) 

TD  (T-5) 

(51,51) 

TD  (T=10) 

(51,51) 

TD  ( T=15 ) 

(51,51) 

DCC  8  bit  (Gradient) 

(51,51) 

MAD  8  bit  (Gradient) 

(51,51) 

TD  (T=5)  (Gradient) 

(51,51) 

TD  (T=10)  (Gradient) 

(51,51 > 

TD  (T=15)  (Gradient) 

(51,51) 

As  can  be  seen  all  methods  correctly  indentified  the 
registration  point,  so  the  best  comparison  of  methods  is  in 
terms  of  relative  peak  sharpness  and  sidelobe  levels. 
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c)  TD  (T=10)  d)  TD  (T=50) 


Figure  5.27  Different  Spectral  Bands,  Gradient  (Horizontal,  Ohio  Files  26,27) 
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Cross-sectional  plots  through  the  peak  are  shown  in 
Figures  5-28  and  5.29  for  the  unprocessed  images  and  in 
Figures  5-30  and  5-31  for  the  gradient  images. 

5-6.3  Different  Scale.  Images  listed  as  HSV  Files  9  and 
10  in  Table  5.1  were  used  in  simulations  of  the  cross 
correlation  of  images  with  different  scale.  Additionally, 
these  two  images  were  for  different  spectral  bands.  Result 
are  summarized  in  Table  5-6  in  terms  of  predicted  registra¬ 
tion  point. 

Peak  profiles  are  given  in  Figures  5-32  and  5.33  for 
the  unprocessed  images.  The  DCC  8  bit,  MAD  8  bit  and  TD 
(various  thresholds)  methods  are  compared.  As  can  be  seen 
from  the  Table  and  Figures,  anamolous  results  were  obtained 
for  the  TD  method  at  low  threshold  values.  This  was  expect 
becuase  of  the  significant  dissimilarity  between  the  two 
images  (different  scale  and  different  spectral  band).  It  i 
significant  that  the  TD  method,  with  a  suitably  large  three 
performs  as  well  or  better  than  the  DCC  and  MAD  methods. 


Figure  5.28  Different  Time  of  Day  (Vertical,  Ohio*File 


a)  DCC  8-bit 


b)  MAD  8-bit 


/"AXIS 

c)  TD  (T=5) 


d)  TD  (T=10) 


e)  TD  (T= 1 5) 


Figure  5.29  Differen 


t  Time  of  Day  (Horizontal,  Ohio  Files  2,10) 
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The  ir.oi.t  significant  result  of  this  ■•ffort  has  b-~-erj  the 
definition  of  a  new  registration  metric  ( the  ti.resnolaod 
difference  method)  which  is  adaptive  to  scene-  content .  lon- 
: arisen  of  the  TD  method  with  DCC  and  MAD  for  the  Scene  1 
and  A  Vertical  arid  horizontal  directions  (with  significantly 
different  frequency  content)  show’s  that  for  the  ideal  case 
(T=  .)  t:.e  TD  method  always  gives  a  perfect  correlation  peak. 

As  the  threshold  (T)  must  be  increased  to  allow’  for  noise 
or  image-  distortions,  some  of  the  advantage  of  the  TD  method 
is  lost.  However,  with  only  a  few  exceptions  (simulations 
which  produced  false  peaks),  the  TD  method  was  as  good  as  or 
better  than  any  other  method.  An  optimum  threshold  for  the 
TD  method  should  be  based  on  some  measure  cf  noise  and/or 
distortion  between  the  two  images  to  be  correlated.  Further 
work  is  recommended  to  provide  the  theoretical  basis  and  a 
practical  measurement  technique  to  establish  an  optimum  value 
for  T  in  a  realistic  application. 

A  single  adaptive  quantization  method  for  the  binary 
correlator  was  shown  to  give  significant  improvement  in 
correlation  peak  sharpness.  Unfortunately  It  showed  high 
sensitivity  to  noise  in  the  images.  If  the  imagery  tc  be 
correlated  has  low  noise  levels  at  higher  spatial  frequencies 
and  if  image  distortions  are  strong  only  at  low  s.  as  ini  fr-.  - 
quencies  then  the  adaptive  quantizer  offers  superior  registra¬ 
tion  performance  over  the  global  quantizer. 
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An  overall  accomplishment  of  this  effort  wa:'  the 
development  of  scene  adaptive  methods  for  improving  image 
registration.  The  two  methods  described  above  offer  poten¬ 
tially  significant  improvements  in  correlator  performance, 
even  though  they  have  some  shortcomings. 

It  is  the  opinion  of  the  authors  that  further  investi¬ 
gation  and  development  of  scene  adaptive  methods  could  make 
pixel-by-pixel  registration  methods  more  accurate  and  less 
sensitive  to  image  distortions.  Follow-on  work  in  this  area 
is  recommended. 
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